On 2-Jul-08, at 12:12 AM, John M McIntosh wrote:

> Ok, well the discussions date from late April 2005.  Some are below.
I also found

On 27-Apr-05, at 7:39 PM, Tim Rowledge wrote:

> In message <a4473bfbe3e462c864d0ef9b9853282e at smalltalkconsulting.com>
>          John M McIntosh <johnmci at smalltalkconsulting.com> wrote:
>
>> question without issues. Lately we added some changes by Andreas for
>> correct weak array handling, some changes to how become: works, and  
>> my
>> work in VM GC statistical data, so I cann't say which is at fault, if
>> any...
>>
> I'm currently hip-deep in similar excrement after merging in the new  
> lowspace-
> process handling and the gc instrumentation/weak pointer stuff. It  
> _looks_ as
> if something is getting twisted in the general gc area since the  
> free block
> size ends up being set to 4. As in '4' not 4k or 4mb, just 4. That  
> quite
> unsurprisingly upsets the sufficientSpaceToAllocate: code and we get  
> an Earth-
> shattering kaboom. Once my head has stopped spinning I'll try the  
> lowspace
> changes without the gc changes.
>
> All to try to make it possible to debug some stupid problem in TK4  
> tree
> walking....
>
Which I think refers pretty directly to the 'secret theft' issue. Ah  
at last - I think this is probably the email where I tried to explain  
the problem-

On 30-Apr-05, at 7:00 PM, Tim Rowledge wrote:

> After having problem trying to debug some TK4 code that blew up with  
> lowspace
> problems but never let me catch and debug, I spent some time adding  
> the
> lowspace-process stuff we recently discussed. I had to make a few  
> alterations
> to match it up with the latest 64bit clean code but no problems with  
> that part.
>
> After building a VM I started testing with some of the methods in
> SystemDictionary 'memory space' - in particular  #useUpMemory. It is  
> perhaps
> fortunate that I did since the other #useUp* methods pretty much  
> work once the
> lowspace process is caught by the vm and passed up to the image.  
> After a _lot_
> of head scratching by John & I we found that with a gazillion tiny  
> objects
> (Links are the smallest possible objects that can exist on their  
> own, plain
> Objects would have to be contained in a collection and so would cost  
> the same 3
> words per object) cause a catastrophic GC explosion. What happens is  
> that
> memory fills up until we get to signal lowspace and then we are in  
> danger.
> Depending upon the exact size of object memory in use the 200kb used  
> as the
> lowSpaceThreshold can be gobbled up in one swallow by the
> initializeMemoryFirstFree: method making sure there is a byte per  
> object that
> survived the markPhase. In using useUpMemory we can get to having 4  
> bytes of
> free space when the next allocate is attempted.... Ka-Boom.
> Expanding the lowSpaceThreshold (along with the VM changes to report  
> the
> process and avoid the accidental problem of interrupting  
> eventTickler) to a
> couple of mb makes it ok on my machine and the threshold can be a  
> lot lower
> with the other tests that create bigger (hence fewer) objects (hence  
> smaller
> fwdTable needs). In the worst case, we could have a very large OM  
> filled with
> very small objects all surviving markPhase; in such a case we would  
> need an
> additional 1/12 of OM available for the fwdTable. So for a 30Mb  
> objectmemory we
> ought to set the lowSPaceThreshold to 30/13 => 2.31Mb + actual space  
> needed to
> run the notifier/debugger etc for reasonable safety. Or hide  the  
> 2.31 Mb away
> so the image never even knows it is there. If you are using virtual  
> memory and
> a limit of 512Mb then you should perhaps secrete 40Mb some where safe.
>
> This assumes that we really need to have one byte per object of  
> course. The
> original rationale was to keep the number of compact loops down to  
> eight (see
> Dan's comment in initializeMemoryFirstFree:) for Alan's large demo  
> image. The
> nicest solution would be to come up with a way to do our GC &  
> compacting
> without needing any extra space. Commence headscratching now... John  
> suggested
> making sure the fwd gets less than the byte-per-object if things are  
> tight, and
> accpting the extra compaction loops.
>
> Good news- with the vm change and 2Mb lowSpaceThreshold I can  
> probably go back
> and find my TK4 problem(s).
>
> Bad news- consider Tweak. With lots of processes whizzing away,  
> merely stopping
> the one that did the allocation and triggered the lowspace is not  
> going to be
> much good. Stopping everything except the utterly essential stuff to  
> debug the
> lowspace will be needed. Probably.

It's really depressing reading mail from back then and even earlier.  
There are long discussions from 2002 about changing the image format  
for version 4, clean compiled methods, closures........

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
Fractured Idiom:- LE ROI EST MORT. JIVE LE ROI - The King is dead.  No  
kidding.