[Newcompiler] [AST] Comments from RBParser

[Andrew Tween]

----- Original Message ----- 
From: "Mathieu Suen" <mathieusuen at yahoo.fr>
To: "The New Compiler, AST, IRBuilder,Closures..."
<newcompiler at lists.squeakfoundation.org>
Sent: Saturday, August 05, 2006 10:00 AM
Subject: Re: [Newcompiler] [AST] Comments from RBParser


> Andrew Tween a écrit :
> > Sorry Math. I confused you with the guy working on the Gutenberg pretty
printer,
> > that was why I mentioned the Dolphin formatter.
>
> YW :)
>
> >
> >> But the question is do we continue to maintaine RBParser or make our
> >> SqueakParser compatible with RB so we can "removed" RBParser(make it as
> >>  an interface like Parser2)?
> >
> > It would be best to have a single parser which serves all the things that
need
> > to parse.
> > Then there is just one parser to maintain.
> >
> >> And I know that you have your parser to for shout...
> >
> > There is no reason why Shout couldn't use a Smacc parser, or RB or something
> > else. The main reason it has a separate parser is because Shout doesn't need
a
> > parse tree, all it needs is a sequence of token ranges. Producing a tree,
only
> > to discard it, slows down the dynamic nature of Shout (since the coloring
> > updates on every keystroke). When I wrote the Shout parser, I wasn't
familiar
> > with Smacc, and attempts at modifying/subclassing the Squeak parser were
> > unsuccessful. Hence the separate Shout parser. But a Smacc parser that
produced
> > a token sequence (no tree) would be better.
>
> I see and understand but
>
> I have tested your parser in shout and I got this results:
>
> SHParserST80 take 17.38528138528139 ms per method
> SqueakParser take 3.852813852813853 ms per method
> Parser take 2.956709956709957 ms per method
>
> Here my test:
> | time i |
> i := 0.
> time := 0.
> time := time + [
> Object selectorsDo:[ :each |
> SHParserST80 new source: (Object sourceCodeAt: each); parse: true.
> i := i + 1
> ]] timeToRun.
> Transcript show: 'SHParserST80 take ', (time / i) asFloat asString, ' ms
> per method', String cr.
>
> i := 0.
> time := 0.
> time := time + [
> Object selectorsDo:[ :each |
> SqueakParser parseMethod: (Object sourceCodeAt: each).
> i := i + 1
> ]] timeToRun.
> Transcript show: 'SqueakParser take ', (time / i) asFloat asString, ' ms
> per method', String cr.
>
> i := 0.
> time := 0.
> time := time + [
> Object selectorsDo:[ :each |
> Parser new parse: (Object sourceCodeAt: each) class: Object.
> i := i + 1
> ]] timeToRun.
> Transcript show: 'Parser take ', (time / i) asFloat asString, ' ms per
> method', String cr
>
>
> Unless I use SHParserST80 in the wrong way?

You need to ensure that a String (rather than a Text) is parsed. i.e.

| time i |
i := 0.
time := 0.
time := time + [
Object selectorsDo:[ :each |
SHParserST80 new source: (Object sourceCodeAt: each) asString; parse: true.
i := i + 1
]] timeToRun.
Transcript show: 'SHParserST80 take ', (time / i) asFloat asString, ' ms
per method', String cr.

I then get this result...
SHParserST80 take 5.70344827586207 ms
per method
Parser take 4.08045977011494 ms per
method

which is a bit less worrying :)

The Shout parser has become progressively slower as more features have been
added. And it parses at a finer grained level than than the standard parser. It
also tries to deal sensibly with incorrect/incomplete source. All of which might
explain the speed difference, I should do some profiling on it.

>
> I think it is the same if you use the tree or not.
> And you can lean on Visitor to color code.

I started out by doing something similar, but found myself extracting the source
ranges from the tree before doing the coloring. The tree was then redundant, so
I thought why bother creating it at all, just record the ranges separately. This
made things faster.
For Shout the important metric is the complete parse/color time. The parser
creates what is needed to make the coloring part faster/easier. The coloring
part changes the RunArray of an existing Text object (rather than creating a new
 Text object from scratch).

>
> >
> >> What do you think?
> >
> > I think that we have too many parsers ;)
> > But we are definately moving in the right direction, and hopefully things
will
> > become unified in the future.
> >
> > Cheers,
> > Andy
> >
>
> Yes I think/hope so to. ;)
>
> I have and idea about adding comments:
>
> It seems that adding comments is like decorating the tree. So why don't
> we do it in the Visitor(we have the source in the tree). Let's say
> ASTCheker?
> May be in a stand alone Visitor?

I think you are right. In the RBParser, the scanner collects up the comments.
And the parser decides which node to add the current set of comments to.


>
> And we can provide a way to chose if we want it or not.
>
> Cheers,
>
> Math
>
>
>
>
>
> ___________________________________________________________________________
> Découvrez un nouveau moyen de poser toutes vos questions quelque soit le sujet
!
> Yahoo! Questions/Réponses pour partager vos connaissances, vos opinions et vos
expériences.
> http://fr.answers.yahoo.com
>
>