[BUG] Squeak 3.4 kernel broken
Andreas Raab
andreas.raab at gmx.de
Tue Feb 4 21:06:38 UTC 2003
Nathanael,
> I didn't have the time for any debugging yet, so I don't know what is
> broken...
I actually mentioned the problem a while ago and it's not trivial. Here's
the relevant part of that message (quoted in full below):
> There's a single problem left at this point which is that you
> can't recompile the entire class hierarchy - it will break at
> Metaclass. This for two reasons. For one, Metaclass class is
> an instance of Metaclass (e.g., "Metaclass class isMemberOf:
> Metaclass" yields true) which means that when we convert the
> _instances_ of Metaclass we also convert the _class_ of
> Metaclass with possibly fatal consequences (and I have _no_
> idea why this has ever worked...) I think that just excluding
> Metaclass' class from the update process should be enough
> here but I'm not certain. The second problem with Metaclass
> is that when you reshape Metaclass (e.g., creating
> NewMetaclass) the meta class created for it (NewMetaclass
> class) needs to be an instance of NewMetaclass for the same
> reason. Not doing so will leave Metaclass with a broken
> instance layout since after reshaping Metaclass its class
> (Metaclass class) will still have the shape of OldMetaclass
> (so to speak) while having the class pointer to NewMetaclass
> (due to #become). I think that some creative use of a
> "multiple new" (e.g., create NewMetaclass class <instance of:
> OldMetaclass>, then create NewMetaclass <instanceOf:
> NewMetaclass class>, then create NewMetaclass2
> class<instanceOf: NewMetaclass>, then create <NewMetaclass2>)
> should do the trick since the only thing that matters is that
> the shape of both NewMetaclass and NewMetaclass class is
> correct (everything else will fixed by #become later).
> Alternatively, a well-placed #primitiveChangeClassTo: might
> work miracles.
>
> If you want to twist your mind around this problem, please do!
Cheers,
- Andreas
> -----Original Message-----
> From: Andreas Raab [mailto:Andreas.Raab at gmx.de]
> Sent: Saturday, September 21, 2002 2:19 AM
> To: 'Nathanael Schärli'; 'schaerli at iam.unibe.ch';
> 'Dan at SqueakLand.org'; 'ian.piumarta at inria.fr';
> 'John.Maloney at SqueakLand.org'; 'Ted at SqueakLand.org'; 'Bill Cole'
> Subject: RE: More ClassBuilder trouble
>
>
> Hi Guys,
>
> I actually got into thinking mode with the problem and it
> turns out we've all been overlooking a couple of things for
> way too long. First of all, some observations:
>
> * Class and Metaclass reshapes: I think that not updating the
> meta-classes when reshaping non-meta classes was a real bug.
> When we create a new non-meta class we already create a
> meta-class for it, and not updating the meta-class where we
> update the non-meta class just brings trouble. The worst
> thing is that it partly breaks the class/meta class
> relationship since the becomeForward: of the old into the new
> class will make the old meta class point to the new non-meta
> class, and the old non-meta class an instance of the new meta
> class. Clearly bad stuff, but in the grand scheme of things
> actually just a minor quibble ;-)
>
> * Obsolete subclasses: What are they?! They are classes that
> still have instances when somebody wanted to remove a class
> from the system. Since they may have instances we need to
> record them just in case their superclass gets reshaped at
> some point. However, from the point of ClassBuilder they are
> just subclasses. There is nothing special about them they
> have _exactly_ the same semantics as regular subclasses have.
> So far so good.
>
> Those of you who have been around ClassBuilder will know
> quite a number of places where the obsolete subclasses were
> managed. When I got into thinking mode one of my first
> observations was that this is a pretty pointless exercise.
> After all, ClassBuilder is *reshaping* classes, not removing
> them, so from the point of updating an old into a new class
> (using become) it simply _cannot_ introduce an obsolete
> subclass. So why bother recording it?!
>
> * Obsolete instances: We go to endless length in order to
> make sure that there are no references to any old instances
> during the reshape process. E.g., we run stuff
> unpreemptively, use become on the instances, and, just at the
> point where we can be _absolutely_ certain there are no more
> references to it, we start remapping those (known to have no
> references) instances into temp instances, use
> #primitiveChangeClassTo: and dance all around the fact that a
> nice fat GC would solve the entire problem.
>
> * 'Super send' pointers: We 'fix' all all of the methods
> while recompiling a class which may send to super in order to
> get super-association right, we dance up and down with this
> stuff, although we will later do a #become: of the old into
> the new class which means that all of this 'fixing' is just a
> plain pointless exercise. Since after the #becomeForward:
> there can be no references to the old class whatsoever, and
> any of those 'fixed' association will point to the new class
> anyways. Which, of course, they would do equally well if they
> would've pointed to the old class - hey that's why we use
> #become, don't we?! ;-)
>
> After I got the above straight down, I started to rip out
> code and try to think what it is we *really* need to do. When
> we mutate a class from old to new then we first mutate all of
> the old class' subclasses (obsolete or not) into subclasses
> of the new one. Fine. Let's assume we're going to do this in
> a way that the old ones simply no longer exists. Then we take
> the old class and want to update the class, its instances and
> possibly its metaclass into the new class, instances and meta
> class. So we need to do an #updateInstancesFrom: which
> converts the instances and will guarantee that there's no
> lasting pointer to any of the old instances (this is a
> critical invariant). At this point, all we need to do is to
> take the old class and meta class and just become them into
> the new class and meta class.
>
> That's it. That's all we ever need. After we've done the
> final #become there are neither any pointers to the old
> instances (known from #updateInstancesFrom:) nor can there be
> any to the old class or meta class (since we just
> forward-became them). So if we do a nice fat GC after
> converting everything we are just done. Period. No need to do
> any of that complex stuff we're doing right now. Neither in
> reshaping nor in preparation of the new classes. Since we
> know we're going to #become them later on we can just create
> new classes, recompile them, update them.
>
> So I thought, and I tried and instantly broke my system ;-)
> Then I spent a day figuring out "what is wrong with #become"
> (since I was absolutely certain that the above _must_ work)
> ... only to find that I had stupidly broken the
> superclass/subclass invariant which we work so hard to
> preserve even for obsolete subclasses. Sigh. But as I said,
> it _must_ work and it does, if you preserve the critical invariants.
>
> Then I went on to remove the GC overhead since although the
> above _greatly_ simplifies the entire reshape process
> (removing four and heavily simplifying three more methods) it
> is somewhat slower due to the extra GC after the #become. I
> found two more possibilities to solve that problem besides
> just that fat GC after each become: We could run the entire
> update of the class hierarchy within a non-preemptive block.
> Since ClassBuilder will not do any "bad stuff" it is known
> that noone else can get a pointer to anything obsolete. But
> still, I didn't like this very much since reshaping the
> entire class hierarchy may run for minutes (even hours on
> slower machines) and not being able to interact, even when
> you absolutely have to, is kinda bad. So I went for the third
> solution which is to assume that generally no other process
> will do "bad stuff at high priority". The basic assumption is
> that no process running above normal priority will use
> reflection techniques to pick up, for example, #allInstances
> or somesuch. Then, we can just put a single well-placed GC at
> the end of reshaping the entire class hierarchy and everything's done.
>
> [Note: If you think that the above 'heuristics' is not good
> enough, we can still switch to any of the other two solution.
> Or use a method that would avoid breaking the class pointers
> - see below]
>
> There's one exception of course. If we reshape the entire
> class hierarchy we may have obsolete Metaclasses when we come
> across Metaclass (since both may be in the tree, the
> metaclasses as well as Metaclass itself). This problem is
> easy to solve by adding "two extra GCs" for the single case
> of reshaping Metaclass itself. That was a reasonable solution
> for me, since how often do we reshape Metaclass in practice?! ;-)
>
> Okay, I have attached the changes for you to play with. These
> changes will work for everything you might try (one exception
> below) IFF your system is in a consistent state to begin with
> (this is important - we've done so many mistakes that there's
> a good chance it may not). The changes simply must work - the
> code is trivial (compared to what was there before almost
> non-existent) and there's nothing that even eventually could
> go wrong if you "play by the heuristics".
>
> There's a single problem left at this point which is that you
> can't recompile the entire class hierarchy - it will break at
> Metaclass. This for two reasons. For one, Metaclass class is
> an instance of Metaclass (e.g., "Metaclass class isMemberOf:
> Metaclass" yields true) which means that when we convert the
> _instances_ of Metaclass we also convert the _class_ of
> Metaclass with possibly fatal consequences (and I have _no_
> idea why this has ever worked...) I think that just excluding
> Metaclass' class from the update process should be enough
> here but I'm not certain. The second problem with Metaclass
> is that when you reshape Metaclass (e.g., creating
> NewMetaclass) the meta class created for it (NewMetaclass
> class) needs to be an instance of NewMetaclass for the same
> reason. Not doing so will leave Metaclass with a broken
> instance layout since after reshaping Metaclass its class
> (Metaclass class) will still have the shape of OldMetaclass
> (so to speak) while having the class pointer to NewMetaclass
> (due to #become). I think that some creative use of a
> "multiple new" (e.g., create NewMetaclass class <instance of:
> OldMetaclass>, then create NewMetaclass <instanceOf:
> NewMetaclass class>, then create NewMetaclass2
> class<instanceOf: NewMetaclass>, then create <NewMetaclass2>)
> should do the trick since the only thing that matters is that
> the shape of both NewMetaclass and NewMetaclass class is
> correct (everything else will fixed by #become later).
> Alternatively, a well-placed #primitiveChangeClassTo: might
> work miracles.
>
> If you want to twist your mind around this problem, please do!
>
> Ah, yes, and two more related notes: The issue with
> MethodChangeRecord is almost non-existent after these
> changes. It came mostly from not updating the meta-classes.
> E.g., CompiledMethods may refer to any classes but will now
> always refer to the "current" class if that class exists. So
> only truly obsolete classes (e.g., those that were removed at
> some point) could be held on by MCR's compiled method. Which
> is good, since it means that the obsolete subclasses now
> _really_ record only truly obsolete subclasses again.
>
> The other note is about #become: changing class pointers. It
> would still be _helpful_ if #become wouldn't change class
> pointers since it would allow us not to worry about the need
> for GC at all (the reason why we need the GC is that all of
> the old class pointers will be broken after become). But it
> is not strictly required. And, it would be _equally_ helpful
> to make #becomeForward: automatically free the the forwarded
> objects, e.g., free the chunks that they used. This might be
> a simpler thing to do, equally effective (no broken
> class-pointers; and we could use one-way become on instances,
> too) and logically, after you're done with a becomeForward:
> the forwarded object is garbage and could be "implicitly
> collected" here.
>
> Cheers,
> - Andreas
>
More information about the Squeak-dev
mailing list
|