[Vm-dev] Re: [Pharo-project] out of memory - cog on windows

Fri Apr 22 21:47:15 UTC 2011

On Fri, Apr 22, 2011 at 3:01 PM, Andreas Raab <andreas.raab at gmx.de> wrote:

>
>  FWIW, the reason for the 512MB limit has to do with Windows memory layout.
> When you're running applications that load dynamic libraries (directly or
> indirectly such as when using ODBC which loads further DLLs) Windows needs
> (sometimes a lot of) address space to map these DLLs in order to load
> them[*]. When the VM starts it grabs MAX_VIRTUAL_MEMORY from the OS which
> will consequently not be available to map DLLs into. We have found that
> depending on the libraries in use and depending on the overall system
> utilization, loading DLLs would fail seemingly "at random" which, after
> further investigation, we were able to track to reserving too much address
> space upfront. We were able to show experimentally, that changing the limit
> from 1GB to 512MB would on some machines make all the difference.
>
> [*] This is true in particular for libraries that create more threads as
> the default Windows policy is to create threads with the stack size of the
> application executable. Thus a 1MB default stack in the application will by
> default create all further threads to be allocated with a 1MB stack size. Of
> course all of this is subject to various other conditions.
>
> In the end we concluded that 512MB is a reasonable size for most apps and
> with 512MB we've never seen these random failures. You can increase the
> limit by recompiling the VM, but if you ship your app in diverse
> environments you should be aware of the potential issues.
>

Thanks Andreas for the explanation. And instead of having to recompile the
VM, we cannot set that from the image side?  or pass as an argument in the
vm execution ? something like the -memory  ?

thanks

Mariano

>
> Cheers,
>   - Andreas
>
> On 4/21/2011 22:44, Alain_Rastoul wrote:
>
>
>
> Apparently, the vm allocates MAX_VIRTUAL_MEMORY and reduces by steps of
> 128M until it
> succeeds.
> so I suppose it could be possible to allocate 2Gb and see how it runs on a
> 1Gb system and if this is not too much stress for
> the system (thinking of the pagefile) ?
> Tudor is your vm a cog or non cog vm ?
>
>
> "Mariano Martinez Peck" <marianopeck at gmail.com> a écrit dans le message de
> news: BANLkTi=Tved13_KL7quOzemOXPnCcfWw+g at mail.gmail.com...
>
>  ------------------------------
>
>
> On Thu, Apr 21, 2011 at 10:21 PM, Tudor Girba <tudor.girba at gmail.com>wrote:
>
>> Hi,
>>
>> Should I to understand that the only way to enable more memory is to
>> recompile the VM? Does that mean that there is no way to pass this
>> information as a parameter like we can on Mac?
>>
>>
> As far as I know you can pass a parameter, but even so, it won't be able to
> allocate more than 512MB.
> I can compile the VM for you with this change in 5 minutes. But I am not
> sure that such simple code would make it work. I think such limit is there
> because of something. Otherwise, it sounds stupid imposing a limit just
> because.
>
>  The problem is that I cannot recompile the VM because I have no access to
>> a Windows machine. Is there one available that provides more memory?
>>
>>
> I don't think so, but start cc'ing the VM mailing list. You'd probably
> receive more help.
>
> Cheers
>
> Mariano
>
>
>> Cheers,
>> Doru
>>
>>
>> On 21 Apr 2011, at 22:09, Alain_Rastoul wrote:
>>
>> > Hi Tudor,
>> >
>> > There is a constant in sqWin32Alloc.h (platforms\win32\vm) :
>> > #define MAX_VIRTUAL_MEMORY 512*1024*1024
>> > you can change it to whatever you want and rebuild the vm,
>> > for exzmple give all the available memory less 256 M .
>> >
>> > HTH
>> >
>> > Regards
>> > Alain
>> >
>> > "Tudor Girba" <tudor.girba at gmail.com> a écrit
>> > dans le message de news: 03B9389F-C719-44D0-B106-2AC78B120F56 at gmail.com
>> ...
>> > Hi,
>> >
>> > We have no specific startUp: methods in Moose. In any case, the issue
>> with
>> > opening the image does not seem to be related to startUp:.
>> >
>> > Is it really true that the Cog VM is limited to 512MB of memory?
>> >
>> > Cheers,
>> > Doru
>> >
>> >
>> > On 21 Apr 2011, at 14:27, Luc Fabresse wrote:
>> >
>> >> Hi Doru,
>> >>
>> >> 2011/4/21 Tudor Girba
>> >> <tudor.girba at gmail.com>
>> >> Hi,
>> >>
>> >>
>> >>
>> >> On Apr 21, 2011, at 14:06, Mariano Martinez Peck
>> >> <marianopeck at gmail.com> wrote:
>> >>
>> >>>
>> >>>
>> >>> On Thu, Apr 21, 2011 at 1:58 PM, Tudor Girba
>> >>> <tudor.girba at gmail.com> wrote:
>> >>>> Hi again,
>> >>>>
>> >>>> I did not say what the problem was :). The problem was that when
>> >>>> opening the image on Windows, he got a Space is low message and the
>> >>>> image was not usable (see attachment).
>> >>>
>> >>> That's weird. Does moose have something on the startup list?
>> something
>> >>> that can be bothering there?
>> >>
>> >> Not that I know of. Is there a way to check this?
>> >>
>> >> Classes should be registered using Smalltlak addToStartUpList: aClass
>> >> Then aClass class>>#startUp: is executed at startup.
>> >> So implementors of #startUp: on Moose classes should give you the
>> answer.
>> >>
>> >> #Luc
>> >>
>> >>
>> >> Actually, using exactly the same process some days ago he produced an
>> >> image of 190MB. This one works just fine on Windows. The only
>> difference
>> >> between the two is the size of the loaded data.
>> >>
>> >> It would be really bad news if the Windows vm would be so severely
>> limited
>> >> in terms of memory.
>> >>
>> >> Cheers,
>> >> Doru
>> >>
>> >>
>> >>>
>> >>> On Mac it worked just fine.
>> >>>>
>> >>>> Cheers,
>> >>>> Doru
>> >>>>
>> >>>>
>> >>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On 21 Apr 2011, at 12:52, Tudor Girba wrote:
>> >>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> I received a question from someone running a 200MB image on Windows
>> >>>>> using Cog 2361.
>> >>>>>
>> >>>>> If I open the image on Mac, it works just fine. Unfortunately, I do
>> >>>>> not have a Windows machine around, and I cannot test but I believe
>> it
>> >>>>> should be solvable by increasing the allocated memory.
>> >>>>>
>> >>>>> On Mac, I would run it with: ./Croquet -memory 1500m
>> >>>>>
>> >>>>> Can anyone help me with the right incantation for Windows?
>> >>>>>
>> >>>>> Cheers,
>> >>>>> Doru
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> www.tudorgirba.com
>> >>>>>
>> >>>>> "What we can governs what we wish."
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> <Space is low.png>
>> >>>>
>> >>>> --
>> >>>> www.tudorgirba.com
>> >>>>
>> >>>> "Yesterday is a fact.
>> >>>> Tomorrow is a possibility.
>> >>>> Today is a challenge."
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Mariano
>> >>> http://marianopeck.wordpress.com
>> >>>
>> >>
>> >
>> > --
>> > www.tudorgirba.com
>> >
>> > "Beauty is where we see it."
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>  --
>> www.tudorgirba.com
>>
>> "If you interrupt the barber while he is cutting your hair,
>> you will end up with a messy haircut."
>>
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
>

-- 
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20110422/357432c3/attachment-0001.htm