lowspace signalling and handling issues

[Andreas Raab]

Hi Tim -

> After having problem trying to debug some TK4 code that blew up with lowspace
> problems but never let me catch and debug, I spent some time adding the
> lowspace-process stuff we recently discussed. I had to make a few alterations
> to match it up with the latest 64bit clean code but no problems with that part.

What am I missing? I don't remember low-space stuff - I only remember 
interrupt-related stuff.

> Depending upon the exact size of object memory in use the 200kb used as the
> lowSpaceThreshold can be gobbled up in one swallow by the
> initializeMemoryFirstFree: method making sure there is a byte per object that
> survived the markPhase. In using useUpMemory we can get to having 4 bytes of
> free space when the next allocate is attempted.... Ka-Boom. 

Well, so don't eat up the memory. There is no reason why 
initializeMemoryFirstFree: would have to reserve that much memory - like 
the comment says the reserve "should" be chosen so that compactions can 
be done in one pass but there is absolutely no such requirement. 
Multi-pass compactions have happened in the past and there is nothing 
wrong with them (in a low-space situation).

> This assumes that we really need to have one byte per object of course. The
> original rationale was to keep the number of compact loops down to eight (see
> Dan's comment in initializeMemoryFirstFree:) for Alan's large demo image. The
> nicest solution would be to come up with a way to do our GC & compacting
> without needing any extra space. Commence headscratching now... John suggested
> making sure the fwd gets less than the byte-per-object if things are tight, and
> accpting the extra compaction loops.

Yes. That's the only reasonable way of dealing with it.

> Bad news- consider Tweak. With lots of processes whizzing away, merely stopping
> the one that did the allocation and triggered the lowspace is not going to be
> much good. Stopping everything except the utterly essential stuff to debug the
> lowspace will be needed. Probably.

Uh, oh. Are you telling me that the "low space stuff" you are referring 
to above actually suspends the process that triggers the low-space 
condition? Bad, bad, bad idea. Ever considered that this might be the 
timer process? The finalization process? Low-space is *not* a 
per-process condition; suspending the currently running process is 
something that should be done with great care (if at all).

Please, don't suspend that process - put it away for the image to 
examine but by all means do NOT suspend it. If you give me a nice clean 
semaphore signal for Tweak to handle a low-space condition I know 
perfectly well what to do but if you just suspend a random process which 
may have absolutely nothing with the low space condition, then, yes, we 
are in trouble (if this were a tweak scheduler process you'd be totally 
hosed).

Cheers,
   - Andreas