which token it can contain.  The way it's done now is that each
	branch is trimed in from the end looking for negated tokens, if I
	understand this right.

What the heck is a NEGATED token?

The only "tokens" in SGML that can begin with a "-" are comments.
An SGML <!DOCTYPE> declaration allows "bare" comments, e.g.,
    <!DOCTYPE --root element-- foo
              --dtd entity-- "ftp://stuff.example.com/dtds/foo.dtd">
An XML <!DOCTYPE> declaration does not.

SGML <!ELEMENT> and <!ATTLIST> declarations also allow comments.
XML ones don't.  (A rather pointless and unhelpful "simplification".)

Neither SGML nor XML allows bare comments in tags.